Skip to content
This repository was archived by the owner on Apr 22, 2023. It is now read-only.

buffer: slow buffer copy compatibility fix#4639

Closed
trevnorris wants to merge 3 commits intonodejs:masterfrom
trevnorris:issue-4633
Closed

buffer: slow buffer copy compatibility fix#4639
trevnorris wants to merge 3 commits intonodejs:masterfrom
trevnorris:issue-4633

Conversation

@trevnorris
Copy link

Fix issue where SlowBuffers couldn't be passed as targets for fast
buffer copy().

Also included small cleanups:

  • Included documentation on how non uint parameters are handled
  • Changed generic Errors to specifics
  • Implemented quicker cc checks for undefined parameters
  • Simplified Buffer checks
  • Fixed buffer copy unit tests

There are many fixes here for Buffer.copy() that, imho, needed to be done, but too many for the v0.8 branch. I'll backport the basic changes and make another pull request for that.

Fix for GH-4633

@trevnorris
Copy link
Author

Just as a side note, the reason for the extensive changes to the unit tests are because each buffer had the same values at the same intervals. So even if the copy failed silently, the asserts would still have passed.

lib/buffer.js Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

target.parent || target?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that was a stupid one. Should have just been placed in the return statement. Done.

@bnoordhuis
Copy link
Member

This should be several small (atomic) commits, not a single big one.

@trevnorris
Copy link
Author

@bnoordhuis Can do. Should each of those be their own PR, or could they be
in included here?

@bnoordhuis
Copy link
Member

Using this PR is fine.

@trevnorris
Copy link
Author

I've tried to split each set of changes as logically as possible.

The first commit has the bare essentials necessary for fixing GH-4633. @TooTallNate requested the change be placed on the v0.8 branch, so I've made sure it can be cherry-pick'd without conflict.

@bnoordhuis
Copy link
Member

Mostly LGTM but I don't know if I agree with changing everything that's not an unsigned int to 0, especially values < 0.

It a) sounds like a potential source of obscure bugs in user code, and b) it could conceivably be used for something useful, like specifying from-end-of-buffer offsets.

@isaacs, @TooTallNate: Thoughts?

@trevnorris
Copy link
Author

@bnoordhuis actually that comment is incorrect. It sets all non-number-y values that may be less than zero to 0, for start and target_start. So it has more or less the same effect as the previous check, except that negative values are set to 0. Just for example:

!(1 > 0) === false
!('1' > 0) === false
!(5.1 > 0) === false
!('a' > 0) === true
!(NaN > 0) === true

And actually, it should be >=. I'll fix that.

As for end, it does basically the same thing except for values greater than this.length. Reasoning behind this one was the previous implementation automatically was set to this.length if end was "oob".

I agree it could be a way to obscure bugs, but don't feel more than it did before. Now the "from-end-of-buffer offsets" is a good point. Though currently all values are ->Uint32Value(), which is a trivial fix, but would also need to be changed. If that's an implementation consideration then I'll put those checks in.

@trevnorris
Copy link
Author

ok. took half a dozen rebases, but think it's taken care of.

@bnoordhuis
Copy link
Member

LGTM. Maybe check with the others if we want to leave room in the API for negative offsets. I like the idea but maybe there are good reasons not to.

@trevnorris
Copy link
Author

@bnoordhuis awesome. so, fixed the space issue I forgot. rebased off most recent master and @isaacs said no go on the negative offsets. think we're good. thanks a lot.

Update: ah crap. changed the commit msg and the code comments, but forgot to change the api. give me 5 to fix it.

@trevnorris
Copy link
Author

@bnoordhuis ok. now I swear it's complete. updated the commit msg, api docs and code comment. also fixed a couple .equal() in the tests that were better set as .strictEqual().

Fix issue where SlowBuffers couldn't be passed as target to Buffer
copy().

Also included checks to see if Argument parameters are defined before
assigning their values. This offered ~3x's performance gain.
Argument checks were simplified by setting all undefined/NaN or out of
bounds values equal to their defaults.

Also copy() tests had a flaw that each buffer had the same bit pattern at
the same offset. So even if the copy failed, the bit-by-bit comparison
would have still been true. This was fixed by filling each buffer with a
unique value before copy operations.
Changed types of errors thrown to be more indicative of what the error
represents. Also removed a few unnecessary uses of the v8 fully
quantified typename.
@bnoordhuis
Copy link
Member

Thanks Trevor, landed in 16bbecc, 49175e6 and cbe3941.

@bnoordhuis bnoordhuis closed this Jan 25, 2013
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small note: I wonder if calling IsUndefined() first is a minor deoptimization. Uint32Value() is only expensive when the value is not an SMI but that should be relatively rare.

@trevnorris
Copy link
Author

@bnoordhuis You're correct. I was focused too much on another aspect. Here are my benchmarks that will help me explain:

                         before        after
fill - fast,fast,noArg:  143,277 µs    174,439 µs
fill - slow,slow,noArg:  308,928 µs    160,340 µs

So at the time I was focused on performance of calling copy against a SlowBuffer with no arguments. For some reason Uint32Value() is much slower when the argument is undefined. But since this is not the primary use case, feel free to revert that segment.

@TooTallNate depending on what @bnoordhuis wants to do here, I know you wanted to cherry-pick the commit to the v0.8 branch. Just a heads up that it was accepted.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants